Background: A cost-effectiveness trial (the Space Bop study) on the added value of botulinum toxin injections (BoNT-A) in the leg muscles, as part of a multimodal intervention for ambulatory children with spastic cerebral palsy in the context of a single distinct cycle of care was performed recently by our group. For a broad set of effect outcomes, we found that BoNT-A had no added value if children received comprehensive rehabilitation. However, this counterintuitive finding was met with scepticism.

Objective: Since several noteworthy facts and experiences were recorded during the course of the trial and the dissemination phase, the aim of this paper was to describe and discuss some crucial aspects of, and barriers to, the Space Bop study, related to context and perspective, design and results, as well as publication and implementation.

Methods: This paper discusses 5 issues: (i) the design, interpretation and presentation of previous research; (ii) the role of one’s own clinical experience and interpretation; (iii) the aims of (BoNT-A) treatment; (iv) conflict of interest, role of industry, and the role of history; (v) optimal treatment modalities and dose–response relationships.

Conclusion: Despite the unambiguous findings from the Space Bop study, several factors hindered acceptance of the results. Awareness of these factors is important when performing rehabilitation research and disseminating and implementing research findings.

Accepted Apr 3, 2020; Epub ahead of print Apr 16, 2020

J Rehabil Med 2020: 52: jrm00059

Correspondence address: Fabienne Schasfoort, Department of Rehabilitation Medicine, Erasmus MC, University Medical Centre Rotterdam, PO Box 2040, NL-3000 CA Rotterdam, The Netherlands. E-mail: f.schasfoort@erasmusmc.nl

^*This article has been handled and decided upon by Chief-Editor Kristian Borg. Henk Stam has not been involved in the decision process.

Recently, we performed a study on the added value of botulinum toxin injections (BoNT-A) for ambulatory children with spastic cerebral palsy (the Space Bop study). The results showed that BoNT-A did not add any value in this context, but this counterintuitive finding was met with scepticism. The aim of this paper is to describe and discuss some crucial aspects of, and barriers to, the Space Bop study. The paper also discusses the design, interpretation and presentation of previous research; the role of clinician’s own experience and interpretation; the aims of (BoNT-A) treatment; conflict of interest, role of industry, and the role of history; and optimal treatment modalities and dose–response relationships. We conclude that several factors hindered acceptance of the Space Bop results. Awareness of these factors is important when performing rehabilitation research and disseminating and implementing research findings.

In ambulatory children with spastic cerebral palsy (CP) in the lower limbs, repeated treatment with botulinum toxin injections (BoNT-A) combined with physiotherapy, casting and/or orthosis as adjunctive rehabilitation interventions is a well-established treatment paradigm. There is consensus that BoNT-A is not a stand-alone intervention (1, 2). The rationale is that leg-muscle injected BoNT-A decreases spasticity by temporarily blocking neuromuscular transmission, which will provide better conditions for adjunctive rehabilitation interventions, and these are then assumed to be more effective. BoNT-A treatment is costly and places a burden (i.e. a toxin is injected, often preceded by some form of anaesthesia) on both the child and his or her caregivers.

Much research has been performed into the effects of BoNT-A treatment for the heterogeneous group of children with spastic CP. The overall conclusion is that BoNT-A has positive effects, e.g. on passive range of motion and on gait (1–7). However, in almost all studies one or more adjunctive interventions, besides BoNT-A, were part of the treatment, i.e. the adjunc-tive interventions were not explicitly withheld. The overall effects of treatment therefore result from the combination of BoNT-A plus adjunctive interventions. Until recently, the specific added value of the BoNT-A component as part of the treatment paradigm had not been investigated, although the need for studying the relative contribution of BoNT-A and other treatment components has been reported several times (5, 8–13).

This was the reason to initiate the SPAstic cerebral palsy: (Cost)Effectiveness of BOtulinum toxin and Physiotherapy (Space Bop) trial. Space Bop was designed as a comparative multi-centre cost-effectiveness trial from a societal perspective, in which “comprehensive rehabilitation only” was compared with “similar comprehensive rehabilitation, but preceded by BoNT-A”. This design allowed conclusions to be drawn about the added value of the BoNT-A component of combined treatment for ambulatory children with lower limb CP. The study included a broad set of outcome measures across different domains of functioning (14), with the primary outcomes being gross motor function, everyday physical activity levels and health-related quality of life.

The Space Bop study was completed, and the comparative effectiveness results on the value of the BoNT-A injections preceding comprehensive rehabilitation have been published in this journal (15, 16). At first glance the results appeared to conflict with current evidence, but this conflict fades when the findings are placed in a broader context.

The current paper shares and discusses some noteworthy facts and experiences that we encountered while performing the Space Bop study, and while presenting, discussing and publishing the results. The paper describes and discusses some crucial aspects of the Space Bop study, related to the design and results, and their dissemination and implementation.

Design

Space Bop was funded by ZonMW (the Netherlands Organisation for Health Research and Development, ZonMW grant number 170995003, HTA funding call 2009), and was originally designed as a multicentre randomized clinical trial (n = 60), in which treatment with comprehensive rehabilitation (only-CR) was compared with comprehensive rehabilitation preceded by BoNT-A injections (BoNT-A+CR). Primary outcomes from the perspective of rehabilitation medicine were gross motor function, everyday physical activity levels, and health-related quality of life, while the secondary outcomes included pain, motor impairments and gait kinematics. In addition, goal attainment scaling (GAS) was used, with goals defined in the activity domain. Data were analysed at primary end-point (3 months) and 6 months follow-up (another 3 months later) using linear mixed models.

During the study, full randomization appeared not to be feasible because of unanticipated, strong preferences of parents/caregivers for one of the 2 treatments (see Schasfoort et al. (15) for more extensive information). Therefore, approval from ZonMW and from the ethics committee of Erasmus MC (NL26738.078.09), the trial was registered in the Dutch Trial Register (Trial NL1576)., the study was altered to a pragmatically designed trial (eventual total 65 participants), in which 40% of the children with lower limb CP were randomized and 60% were assigned to the treatment arm their parents/caregivers preferred. This resulted in 41 children in the BoNT-A+CR group, and 24 in the only-CR group.

The findings from Space Bop were clear and straightforward and have been described in 2 comparative effectiveness papers (15, 16): no differences between interventions were found at 3 and 6 months, i.e. with comprehensive rehabilitation there was no evidence for the added value of BoNT-A on any of the primary and secondary outcomes, nor on GAS. The total cost per child for treatment up to primary end-point at three months was significantly higher for BoNT-A+CR than for only-CR (8,963 vs 6,182 euro, p=0.001). Because of the pragmatic design, there were imbalances in size and some baseline patient characteristics between the (larger) group that received BoNT-A+CR and the (smaller) group that received only-CR. Because this could potentially affect the validity of findings, baseline comparability was described in detail (15, 16), and several additional analyses were performed that focused on the effects of potential sources of bias. These analyses did not indicate any confounding effect.

Dissemination

After data analyses, the results were presented at several national and international conferences, and the 2 comparative effectiveness papers were submitted (15, 16). There was a lengthy (re)submission phase, during which many comments and, mainly design-related, immediate rejections were addressed. The fact that the study was not fully randomized was the main issue, but other methodological challenges known to occur in the evaluation of rehabilitation interventions, such as the effect of adjunctive interventions and expectation bias, were also topics of discussion (17).

In this period, both invitational and open-access project meetings were organized to inform children, parents, informal caregivers, medical specialists, the-rapists and other interested stakeholders. Especially during the meetings with clinicians, who were mainly rehabilitation specialists, 3 distinct groups were recognized: (i) believers: “I’m not surprised by the results, and I will change my treatment”; (ii) non-believers: “the results are not correct, especially given the methodological shortcomings (not being fully randomized)”; and (iii) the intermediate group: “the results might be correct, but there still might be reasons to apply BoNT-A (e.g. for other subgroups with CP, other treatment goals), the indication for BoNT-A treatment could probably be narrowed”.

In short, the results of the Space Bop study gave rise to much discussion and, perhaps also, some confusion. The results were not automatically and immediately accepted and implemented. It is known that a general tendency exists to reject new knowledge, especially when it contradicts existing beliefs (known as the Semmelweis reflex), as described by Mortell and colleagues (18). It is also known that the implementation of scientific evidence and guidelines in clinical practice is a complex process, and that the determinants of a successful strategy are still not fully understood (19). Therefore, it might be expected that the acceptance of our counterintuitive results and discontinuation of the wide use of BoNT-A treatment would be a difficult and time-consuming process. What surely contributed to the scepticism we encountered was the fact that Space Bop was the first study to explicitly focus on the added value of BoNT-A as part of a multimodal intervention in the context of a single complete cycle of care, and the above-described methodological issues that we never dismissed. Nevertheless, we feel that other factors have played, and still play, a dominant role in the phase of acceptance and dissemination, as discussed below.

Interpretation and presentation of previous research

Space Bop was the first study that specifically focused on the added value of BoNT-A preceding multimodal comprehensive rehabilitation in comparison with an equally intensive multimodal comprehensive rehabilitation intervention without BoNT-A, but it was not the first study on the effects of BoNT-A combined with comprehensive rehabilitation. As already elaborated on in the Introduction, the overall effects of treatment include the combination of BoNT-A plus adjunctive interventions (Fig. 1a). Strikingly, in most of the previous studies only the BoNT-A part was put forward as the effective component, thus disregarding the (potential) contribution of the modalities from the comprehensive rehabilitation part, such as physiotherapy, casting and orthosis (e.g. (5, 6, 20–24) (Fig. 1b). For instance, in the latest Cochrane review, all the comparisons made are presented as “BoNT-A” vs “other treatments” which does no justice to the actual comparison that has been made (4). As stated, Space Bop was the first study that aimed at this comparison (Fig. 1c).

When the literature is critically appraised, there are a number of indications that the relative contribution of factors other than BoNT-A to overall effectiveness of combined treatment packages may have been underestimated (2, 13, 22, 25–27). In short, a stronger relative contribution of factors other than BoNT-A may equally well explain improvements, especially in view of the longer-term effectiveness (16). It should be noted that the quality of the evidence for BoNT-A treatment of lower limb spasticity was low to very low in the most recent Cochrane review (4). In addition, from Norwegian registry data it has been reported that intensive training was the only intervention factor associated with enhanced gross motor progress (28, 29).

The main issue here is how results, almost without exception, have been presented in especially titles and abstracts of publications: namely as stand-alone BoNT-A effects. Such presentation is probably based on an implicit assumption that BoNT-A is the most active component in combined treatment packages (15), but does not match the comparison that has actually been made and is also not in line with the consensus that BoNT-A should be combined with other interventions in which a number of different clinicians is involved in a single treatment cycle. It has been reported that, in many cases, the interventions under comparison largely remained undefined black boxes (10, 15, 16, 30). This makes it unclear whether the firm conclusions regarding effectiveness of BoNT-A treatment have been drawn from a comparison of “apples and oranges” rather than from a fair comparison. For a proper interpretation of findings it is important to take this into account: presenting (differences in) effects (between groups) as stand-alone BoNT-A effects (especially when it is unclear whether children had specifically refrained from other interventions) may be quite mis-leading and may place readers of those papers, and possibly even reviewers or editors, on the wrong track.

The role of one’s own clinical experience and interpretation

In the Netherlands, rehabilitation specialists and neuro-logists lead the treatment of children with spastic cerebral palsy. After 1-day outpatient BoNT-A treat-ment in the hospital, the children visit a physiotherapist for a period of 3 months, and when indicated, they also receive casting and/or orthoses. Besides the phenomenon described above, it seems to be possible, or even logical, that medical specialists consider their role and contribution as the most important part of successful treatment. In the social psychology this is known as “self-serving bias” (31). Thus, when they notice or are informed about the positive effects of treatment, they might tend to experience their contribution as most important, especially in case they are not directly involved in the other (in this case mainly the physical therapy component) part of treatment. In contrast, when a negative effect is noted, it is tempting to attribute failure to others involved in the full cycle of care, or to refractory clinical practice in general. This mis-estimation or over-estimation of one’s own role might have contributed to interpreting the Space Bop results as “counterintuitive”.

Shift in aims of (BoNT-A) treatment

During the course of the Space Bop study we observed interesting shifts in the discussion about the aims of treatment cycles. The a priori defined primary outcomes from the broad set of outcome measures were analysed first: gross motor function, everyday physical activity levels and health-related quality of life. When those results were negative regarding the added value of BoNT-A, “clinicians” (we are generalizing here) responded that those outcomes were not most important, that some of the secondary outcomes and GAS were more relevant, and that effects on these outcomes would be more logical given the BoNT-A working mechanism. When these latter outcomes (i.e. functional leg muscle, strength, passive range of motion, angle of catch, CP-related pain, walking speed, kinematic gait parameters, goal attainment, and proxy-reported general functioning) also showed negative results, it was reported that BoNT-A would certainly have other, long-term effects, such as the prevention of contractures, possibly only within a subgroup of ambulatory children with CP. We will not discuss the rationality of these assumptions, and BoNT-A might have relevant long-term effects, but we only want to emphasize the need of being precise on the initially defined research questions and outcomes, and not change the reasoning during the project. Being clear about the aims of treatment is also of major importance in clinical practice: children and their parents may expect to be precisely informed about the goals of treatment and the effects that can be expected.

Conflicts of interest, role of industry, and history

Space Bop was funded by ZonMw, a Dutch governmen-tal organization for health research and development. Space Bop therefore was not industry-initiated, and was completely independent of any commercial influence. There are indications that industry-initiated trials generally result in more positive (read: biased) results and conclusions (32). Even non-industry-initiated trials might be affected by dependencies between industry and medical specialists. In addition, industry-initiated trails generally have different goals; they focus on effect size (=large contrast between interventions), registration of a drug, and safety/efficacy. Therefore, as we described earlier, disentangling the relative contribution of the different modalities of treatment cycles is not of primary concern (16).

For a long time now, undoubtedly with positive intentions, the application of BoNT-A has been promoted, and, since the turn of this century, BoNT-A treatment is reimbursed by health insurance companies in the Netherlands, although paediatric use of BoNT-A other than for the gastrocnemii muscle complex is officially off-label (33). BoNT-A treatment is provided by trained and specialized rehabilitation specialists and/or neurologists, and it is imaginable that those medical experts are reluctant to abandon this intervention and experience the risk of having to abdicate their “professional child”, especially since many feel it benefits the children.

Optimal treatment and dose response

One of the critical comments on our study was related to the applied doses of BoNT-A, which were sometimes considered too low. However, the treatment modalities in the Space Bop study were carefully defined, and in agreement with the consensus of that moment (1, 2, 34, 35), and, if needed, adapted by the latest scientific evidence. However, in the field of BoNT-A the number of dose–response studies is relatively limited and the available studies are difficult to interpret (7). Because of the challenges that are related to clinical trials in chronic (paediatric) rehabilitation populations, such as CP (13, 17, 36), for the future it is probably advisable to not only focus on randomized controlled trials, but also on studies that provide better insight into working mechanisms and dose–response relationships, which also account for the adjunctive (rehabilitation) interventions that are part of the full treatment cycle (4). This latter topic (i.e. the moderating effect of intensity and content of the different modalities applied, including the effect of guideline adherence for physical therapy periods) is the focus of a paper in preparation by our group.

Another discussion was about the mode of application of BoNT-A (using ultrasound, electro, manual as guidance); as long as the interplay of complex combined interventions is not taken into account in effectiveness research it is difficult to make a statement about this. Also, regarding recent BoNT-A safety discussions, it is important to always balance potential benefits of BoNT-A against possible negative long-term effects, such as muscle atrophy and fibrosis, and their subsequent implications for future functioning of children with CP (37).

Conclusion

The conclusions of the Space Bop study, although we recognize the suboptimal methodology regarding randomization, were clear: in ambulatory children with CP there was no evidence for an added effect of leg-muscle injected BoNT-A prior to comprehensive rehabilitation on a large set of outcomes measures. However, acceptance of these findings appeared to be difficult. We do not question the working mechanisms of BoNT-A, nor do we want to overestimate the clinical implications of our work. Nevertheless, we think that the results should be recognized and should be considered carefully, with the topics of the current report in mind.

Implementing the results of scientific research is a societal responsibility of researchers, but this also appears to be complex and needs specific expertise. Barriers to this process can have a rational background, but it is important to be aware that irrational and un-desirable barriers might be even more prominent. With the description and discussion of our Space Bop study as an example, we hope to contribute to elucidating the role and effect of these barriers.